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This paper presents a framework intended to link the following assessment development concepts into a systematic framework: 
evidence-centered design (ECD), scenario-based assessment (SBA), and assessment of, for, and as learning. The context within which 
we develop this framework is the English language arts (ELA) for K-12 students, though the framework could easily be applied to cover 
reading, writing, and critical thinking skills from pre-K through college. Central to the framework is the concept of a key practice, 
drawn from constructivist learning theory, which emphasizes the purposeful social context within which skills are recruited and 
organized to carry out complex literacy tasks. We argue that key practices provide a key link between existing CBAL™ ELA learning 
progressions (defined as part of a student model for literacy skills) and the structure of well-designed SBAs. This structure enables us 
to design assessments that model a key practice, supporting the systematic creation of task sequences that can be used to support both 
instruction and assessment. 

Keywords CBAL™-, cognition; cognitively based; scenario-based; assessment; assessment design; English language arts; 
constructivism; reading; writing; critical thinking; evidence-centered design; domain analysis; student model; task model; evidence 
model; key practice; learning progression 
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In a series of papers, Deane and colleagues (Deane, 2011; Deane, Sabatini, & O’Reilly, 2013; Sabatini, O’Reilly, & Deane, 
2013) described the theoretical and conceptual foundations of an English language arts (ELA) competency model and 
a set of hypothesized learning progressions associated with this model. This body of work is the ELA portion of the 
Educational Testing Service (ETS) research initiative known as Cognitively Based Assessment of, for, and as Learning 
{CBAL™) learning and assessment tool (Bennett, 2010,2011,2013; Bennett, Kane, & Bridgeman, 2011; Fu & Wise, 2012). 
The most recent framework is publicly available at the CBAL wiki 1 and in related reports (Deane, 2011; Deane et al., 2008; 
Deane et al., 2013; Sabatini et al., 2013). 

The process by which the CBAL ELA framework and existing CBAL ELA assessments were developed falls solidly 
within the tradition of evidence-centered design (ECD; Mislevy, Almond, & Lukas, 2004; Mislevy & Haertel, 2006), which 
emerged in large part from efforts to extend the theory of assessment design to cover nontraditional assessments, including 
simulations and other performance assessments (see, e.g., Frezzo, Behrens, & Mislevy, 2010; Mislevy, 2013; Mislevy et al., 
2003; Mislevy, Steinberg, Breyer, Almond, & Johnson, 1999; Rupp, Gushta, Mislevy, & Shaffer, 2010; Shute, Masduki, & 
Donmez, 2010; West et al., 2010; Zalles, Haertel, & Mislevy, 2010). 

In effect, the 2011 CBAL ELA framework is an extended example of how educators and researchers can move from 
domain analysis to building a conceptual assessment framework (CAF) within the ECD tradition, combining a student 
model (specifying what is being assessed), a task model (specifying what tasks are to be used to measure student model 
variables), and an evidence model (specifying how each task type provides evidence for particular student model vari¬ 
ables). The CBAL ELA framework focuses on defining categories of activities and skills and mapping these onto learning 
progressions (scales that define hypotheses about a sequence of qualitative shifts in performance that may be observed 
as a skill or concept is mastered), which are then used to define evidence and task models. The framework provides a 
comprehensive description of the full developmental continuum of ELA knowledge and skills from preschool through 
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college and career readiness, with links to the Common Core Standards (Common Core State Standards Initiative, 2010a, 
2010b). Since 2007, a variety of pilot assessments have been developed based on this evolving model. 

However, using the 2011 CBAL ELA framework to communicate the connections between good instructional practice, 
CBAL assessments, and the emerging evidence base from the cognitive and learning sciences literature to a broader audi¬ 
ence of educators has been challenging. Making these connections clear is a key goal of the CBAL initiative, which seeks to 
build an assessment system that helps teachers make sound educational decisions toward enhancing their students’ ELA 
knowledge and skills. 

This report therefore expands on the prior CBAL framework in order to introduce a critical element that links assess¬ 
ment design with instruction. This linking element, the concept of a key practice, derives from the general conceptual 
framework of activity system theory (Bazerman, 2004; Hung & Chen, 2002; Jonassen & Rohrer-Murphy, 1999; Vygotsky, 
1978). It fills a critical role within the CBAL framework because it helps to link the ELA competency model to another 
element that has been extensively explored in CBAL research: scenario-based assessments (SBAs). In CBAL, SBAs com¬ 
bine two important features: They consist of a series of discrete tasks designed to measure targeted skills and lead up to 
a culminating task that measures the integration of skills. The coordinated sequence that results simulates a purposeful 
scenario within which one or more agents take a meaningful series of steps toward an overall goal or purpose. The result¬ 
ing assessment resembles a performance task (because it provides a context and purpose for all the specific tasks that it 
contains), but it has many of the advantages of a traditional standardized test because it measures targeted skills through 
a series of separate items. CBAL researchers have been developing the case for SBAs for some time, both within ELA 
(O’Reilly & Sabatini, 2013; O’Reilly & Sheehan, 2009; Sheehan & O’Reilly, 2012) and in other content domains (e.g., Graf, 
Harris, Marquez, fife, & Redman, 2009; Liu, Rogat, & Bertling, 2013). 

CBAL SBAs deploy techniques that effectively integrate cognitive and learning sciences insights within a rigorous 
assessment design. In particular, SBAs are designed to be more user friendly than traditional multiple-choice test designs 
in that they incorporate a meaningful purpose for reading and writing and model instructional strategies and tasks repre¬ 
senting best instructional practices while employing technology to support learning and motivation using strategies from 
learner-centered design (Soloway, Guzdial, & Hay, 1994). In this report, we argue that key practices help define the ELA 
construct in a way that makes it much easier to select appropriate scenarios for CBAL SBAs and to design their item-level 
content. 

The report is structured as follows. To start, we define and describe a set of 10 key practices for the ELA. Then, we 
discuss how these key practices connect with the existing CBAL framework and with related frameworks, such as Reading 
for Understanding (RLU; cf. O’Reilly & Sabatini, 2013; Sabatini et al., 2013). Next, we examine how an analysis of key 
practices can inform the structure and sequence of tasks in CBAL SBAs. In particular, we argue that a key practice is a 
class of literacy activities that use similar methods to accomplish similar goals and that an SBA as defined here can be 
interpreted as a somewhat idealized instantiation of the sequence of activities that define a key practice. Then, we propose 
a specific technique that leverages the structure of key practices to support creation of SBA blueprints. In particular, we 
argue that typical sequences of activities within a key practice can be used to define prototypical scenarios. Context plays 
a critical role in defining such scenarios. The scenario structure and the key practice suggest a class of relevant contexts to 
which the skills in question are applicable. This link between key practice, associated skills, and applicable contexts should 
help the student (and teacher) make appropriate links to the conditions of use appropriate for the key practice and skills. 
Such prototypical scenarios can then be linked to learning progressions that describe the skills required by each task, 
after which levels in each learning progression can be mapped onto specific item designs. This sequence of inferences 
provides us with a validity argument for an SBA blueprint in which each item can be linked back to a specific skill, and the 
relationship between skills and tasks is implicit in the scenario structure of the SBA. finally, we illustrate how the ideas we 
have developed can be applied in four specific areas: research and inquiry, early reading, informational text processing, 
and argumentation. 

Within an ECD framework, the relationship among key practice, SBAs, and learning progressions not only helps with 
domain analysis, it also helps to specify the CAL 

• The association between each key practice and a set of learning progressions helps us to define a student model. We 
expect that the model will enable educators to evaluate student performance overall, by student ability to carry out 
the full integrated practice, and with respect to specific skills, where specific learning progressions define specific 
student model variables. 
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• The learning progressions do most of the work needed to define the evidence model. However, the key practice 
frames the evidentiary arguments. A specific skill—such as the ability to recognize supporting evidence for an 
argument—matters because it has a clear supporting role within the larger practice. 

• Finally, the link between SBAs and key practices plays a key role in defining the assembly model. Using an SBA 
imposes the constraint that the assembled test must respect the structure of the key practice(s) being assessed. This 
principle works to guide both the choice and the sequencing of tasks. 

Key Practices in the English Language Arts (ELA) 

Our conception of the ELA is grounded in constructivist approaches to learning more generally (Brown, Collins, & 
Duguid, 1989a; Spiro, 1988; Vygotsky, 1978) and in studies of the development of expertise in particular. Particular forms 
of expertise can be viewed in terms of activity theory (Engestrom, Miettinen, & Punamaki, 1999) and can be analyzed as 
activity systems. Each activity system is an established pattern of social interactions, or practices, in which particular skills, 
tools, and forms of knowledge are critical for full participation. Developing expertise in a skill emerges naturally from par¬ 
ticipating in the practices, possibly after a form of cognitive apprenticeship (Brown, Collins, & Newman, 1989b; Gee, 2001, 
2007; Hung & Chen, 2002; Jonassen & Rohrer-Murphy, 1999; Rogoff, 1990). When we apply this kind of thinking to the 
ELA, it follows that particular genres, or types of text, are motivated by the role they play in a larger system of practices 
(Bazerman, 2004; Russell, 1997). At a finer grain size, specific discourse communities may foster a series of specific prac¬ 
tices, including discipline-specific genres that require specific disciplinary skills (Geisler, 1994; Graves, 1991; Hunt, 1996; 
Rouet, Favart, Britt, & Perfetti, 1997; Voss & Wiley, 2006). For instance, a complex of activities associated with rational 
argument includes oral discussions and debates, ongoing debates in online forums, and formal argument in academic 
settings. This complex of activities includes the logical analysis of arguments, and associated rhetorical moves like coun¬ 
terpoint and rebuttal, but it includes a wide range of other skills that contribute to the underlying goal—a collaborative 
reasoning process in which people seek to convince each other of the truth of their claims. 

The shift toward key literacy practices as an assessment framework has two important implications. First, we see the 
construct of an ELA assessment as including not only the individual skills and knowledge pieces necessary to accomplish 
reading and writing activities but also the understanding and appreciation of these activities as part of a meaningful 
social practice to achieve a larger communicative goal. Although this emphasis of the social aspect of literacy has been 
present in earlier CBAL ELA frameworks (Deane, 2011; Deane et al., 2013; O’Reilly & Sheehan, 2009), it is elevated as a 
guiding principle for thinking about ELA assessment. We intend the framework to measure the extent to which a student 
understands the purpose and process of key literacy practices as much as the knowledge pieces and subskills to accomplish 
this goal. 

The second implication has to do with the flip side of the same coin. If one views literacy development (particularly 
in a typical K-12 classroom) as an apprenticeship, in which students observe, practice, and demonstrate literacy skills 
in a (explicitly or implicitly) scaffolded environment, then one should not be afraid to model the scaffolding in literacy 
assessments, particularly if it is intended that the assessments will inform learning and teaching (Bennett, 2010; Deane, 
2011; Deane, Fowles, Baldwin, & Persky, 2011). This point is articulated later in this report (see section The Link Between 
Key Practices and Scenario-Based Assessments). 

The domain analysis problem for an assessment in the ELA, including reading and writing, thus becomes one of defin¬ 
ing the key practices in which literate individuals are expected to be able to participate. Such practices must be defined 
specifically enough that they can be taught and assessed yet generally enough to cover a range of more specific expert 
practices, both academic and professional. If these practices are defined clearly enough, they will also define what types of 
scenarios can be included reasonably in an SBA. In other words, each key practice includes sequences of related activities, 
and expertise includes both the ability to handle each activity on its own and the ability to determine flexibly which com¬ 
bination of these activities is required for successful performance in the range of situations for which the key practice is 
appropriate. In other words, literacy skill is defined precisely by the ability to generalize the routines and procedures that 
define a key practice across all of the contexts in which they are appropriate. 

The goal of this section is to define a set of key practices in the ELA and to describe how these practices constitute 
a domain analysis that can be used to generate SBAs. We view these key practices as consistent with existing ELA stan¬ 
dards frameworks such as CCSS 2 and with earlier CBAL reading and writing literature reviews. However, our approach 
is distinguished by its focus on integrated practices — on the functional interconnections among specific skills and their 
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integration to support meaningful literacy activities. This focus means that we are not interested in measuring reading, 
writing, critical thinking, or other specific literacy skills as if they were isolated sets of measurable skills. We focus instead 
on how various activities integrate to form meaningful practices and on the way specific skills work in concert to support 
the rich range of literacy practices targeted by standards for college and career readiness. As we have developed this set 
of key practices, we have drawn heavily on the literature on genre (e.g., Bazerman, 2004), reading and writing cognition 
(Hayes, 2006; Oakhill, Cain, & Elbro, 2014), and sociocultural theories of reading (Gee, 2001, 2007) and writing (Prior, 
2006). As we did so, we tried to identify recurring themes and patterns — structures of literacy activity that recurred across 
a wide range of social contexts and applied to the practices associated with a variety of specialized text genres but also 
linked strongly to existing educational standards. 

As a result, alignment between key practices (as we conceive them) and standards like the ELA portion of the CCSS 
requires attention to the relationship between reading and writing. For instance, argumentation is addressed twice in the 
CCSS — once in reading, once in writing. In the CCSS, research is treated as a writing skill, despite the core role that reading 
skills play in conducting research, and so forth. Instead of measuring aspects of the same literacy practice separately under 
different standards, we suggest that assessments may be designed based on a more integrated conception of the construct. 

A specification of key practices for ELA should consider (a) how skilled practitioners or experts engage in literate prac¬ 
tices in academic and professional settings, (b) what component skills are necessary to support performance of integrated 
practices and should be targets for assessment, and (c) what current theories of cognitive processes and cognitive devel¬ 
opment have to say about the emergence of literacy skills. With these considerations in mind, we introduce a provisional 
set of key practices that we believe are most relevant for K-12 ELA assessment. This is not meant to be an exhaustive list 
nor the only way to parse the largely overlapping literacy practices in academic and professional contexts. 

The sections that follow describe 11 key practices that cover essential literacy (ELA) skills required for college and 
career readiness. As illustrated in Figure 1, we cluster the key practices into three more general categories: fundamental 
literacy, model building, and application. 

Fundamental Literacy Practices 

Fundamental literacy practices are the practices that support entry into literate English-speaking communities. In K-12 
education, they are most strongly emphasized in K-4, though they continue to develop in higher grades, where they are 
more and more closely intertwined with a variety of other, more specific literacy practices. Typically, later forms of literacy 
are embedded within fundamental literacy practices, but in a more scaffolded form, just as acquisition of later forms of 
literacy presuppose mastery of the fundamentals. 

Communicate by Speaking and Listening (Oral Language) 

This class of practices is based on fluent control of English language skills needed to participate effectively in literate 
communities. In younger age groups, critical skills include those needed to participate in conversations around language 
and print, which differ from everyday language uses and are important predictors for literacy acquisition (Snow, 1983, 
1991). In higher age groups, critical skills include competency in a variety of oral interactions, including group discussion 
and oral presentations, and require development of advanced language skills that normally only emerge after immersion 
in a literate community where sustained reading and writing are normal practice. 

Read Silently and Aloud (Early Reading) 

This class of practices focuses on the skills needed to embed reading in other cultural practices. However, it involves more 
than the development of print and decoding skills; the practices that support early reading include a variety of situations 
in which oral reading is embedded (such as shared reading of picture books). Over time, the emphasis gradually shifts 
from shared, oral reading to silent reading without immediate social support. 

Write Down Words and Ideas (Early Writing) 

This practice focuses on the skills needed to embed writing in other cultural practices. However, it involves far more than 
the development of handwriting, spelling, and so on; the practices that support writing development focus on socially 
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Fundamental Literacy 



Model-Building 



Application 



Figure 1 Key practices for the English language arts (ELA). 


natural, age-appropriate contexts for written communication such as notes, letters, text messages, or e-mails. As this skill 
matures, emphasis shifts from transcribing what others say to expressing one’s own ideas in written form. 

One way to conceptualize the fundamental literacy practices is that they start with oral interaction and move students 
to the point at which written practices can substitute for oral equivalents, for instance, substituting written questions 
and answers for oral questions and answers, and thus introducing quizzes or homework as substitutes for direct oral 
interaction with a teacher assessor. This process is roughly compatible with the so-called simple view of reading (Hoover & 
Gough, 1990). With modal native speakers, who have the language and conversational skills needed to enter productively 
into school culture, the early elementary curriculum focuses on building print skills necessary to use written language 
fluently; increasing metalinguistic awareness, which will be crucial in more complex forms of literacy; and scaffolding 
a richer understanding of more advanced literate practices. However, instruction must also support students who enter 
school without mastery of conversational English or whose everyday experience does not prepare them to handle the full 
range of oral interactions needed to scaffold more complex types of literacy. 


Model-Building Practices 

These practices require readers and writers to build mental models of text and context. They presuppose fundamental 
literacy and focus primarily on the relationship between form and content. They begin to build up the ability to use 
and create extended texts that contain far more information than any one person can easily organize or explain without 
supporting tools and strategies. In K-12 education, this class of practices is a focus in Grades 3-8, though these practices 
are embedded in scaffolded forms in earlier, emergent practices and play a critical role in supporting applied forms of 
literacy. 
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Develop and Share Stories and Other Social Understandings 

This class of practices requires mastery of skills and strategies needed to understand and communicate about real and 
imagined social situations. It includes the ability to understand and tell stories, whether orally or in writing. It also includes 
the ability to grasp and communicate about real-world social situations using many of the same abilities needed to under¬ 
stand narrative. 

Build and Share Knowledge From Text 

This class of practices requires mastery of skills and strategies needed to acquire knowledge from a source text and share it 
with others, whether orally (by asking and answering questions or providing and listening to explanations) or in written 
form, by reading and writing about informational texts, even when they are hard to understand. 

Draft, Revise, Edit, and Publish Texts 

This class of practices requires mastery of skills and strategies needed to participate in the process of producing texts for 
publication, including a variety of roles other than reader or author of on-demand text, such as reviewer, reviser, editor, 
or proofreader. It includes a range of reading and thinking skills necessary to evaluate writing quality and manage writing 
processes. 

The model-building practices are concerned both with creating a richer understanding of (and ability to work within) 
text genres and with creating richer modes of interaction that are enabled by the ability to work fluently with text. Stories 
provide a rich entry point into social worlds and make it possible both to build up an understanding of narrative and liter¬ 
ary genres and to develop understandings of how people interact in a range of social situations that go well beyond most 
students’ daily experience. Informational texts provide a window to domains of knowledge to which students may never 
have been exposed, while also modeling methods for communicating information to others. In combination with early 
work on writing and editing skills, these practices help students to build the background knowledge and skills necessary 
for them to participate in literate culture not only as consumers of text, but also as producers. 

Applied Practices 

Applied practices are those in which literacy skills are applied to do intellectual work (such as critical analysis, argumenta¬ 
tion, scientific and/or historical inquiry, and analysis and management of complex projects and proposals). In the current 
state of American K-12 education, applied practices may often emerge as primary curricular targets in middle school but 
come into focus in the high school grades. Under current conditions, many students will not achieve high levels of exper¬ 
tise in these practices until college or beyond, although early elementary students may be capable of developing significant 
skills in these areas, especially in oral contexts. 

Analyze Craft and Literary Elements 

This class of practices involves mastery of the skills needed to analyze the choices made by a writer and draw out their sig¬ 
nificance and implications. It includes rhetorical analysis, analysis of literary techniques, and consideration of the relation 
between a text and its context. 

Build and Justify Interpretations 

This class of practices involves mastery of skills and strategies needed to participate in a discourse where reaching agree¬ 
ment on the meaning of a text is an important task, including the ability to analyze and justify interpretations and select 
between alternate readings on the basis of textual evidence. 

Discuss and Debate Ideas 

This class of practices involves mastery of skills and strategies needed to consider an idea from multiple perspectives and 
assemble arguments to favor one position over another, whether orally (by participating in discussion and debate) or in 
written form (by creating, evaluating, and rebutting written arguments). 
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Conduct Inquiry and Research 

This class of practices involves mastery of skills and strategies needed to participate in a research community, including the 
abilities to gather, evaluate, and synthesize information from multiple sources; to conduct inquiry and experimentation; 
and to present information one has learned from sources in appropriate forms and formats. 

Propose, Review, Recommend, and Evaluate 

This class of practices involves mastery of skills and strategies needed to plan and carry out literacy activities designed 
to solve a problem or achieve a shared goal. This set of practices presupposes social or interpersonal skills needed to 
support effective collaboration but focuses on the specific literacy skills needed to perform effectively in a collaborative 
context, such as setting goals, defining and applying standards or success criteria, developing and evaluating proposals or 
recommendations, and reporting project results. Ultimately, these kinds of applied practices correspond to the kinds of 
skills that define the social contexts that make reading and writing meaningful. These practices are particularly critical in 
developing so-called 21st-century literacy (New Media Consortium, 2005). 3 Especially in oral forms, all of these practices 
may be introduced very early in the curriculum, but as fundamental and model-building skills mature, applied practices 
become more and more the focus of instruction, until they define the literate practices that are central to college and 
career. 

Developmental Issues 

As our discussion implies, there is (to some extent) a developmental relationship among the key practices, parallel to clas¬ 
sic formulations such as the distinction made by Chall (1983) between learning to read, reading to learn, and reading to 
do. Before one can apply literacy skills to solve complex problems, one has to be able to build a mental model (i.e., com¬ 
prehend) from texts and communicate that mental representation through writing, which in turn requires foundational 
skills such as word recognition, spelling, and language comprehension. Importantly, more advanced key practices often 
subsume one or more earlier key practices. In other words, successful participation in higher level key practices typically 
requires fluency and flexible cognitive control over lower level literacy practices. 

Although it is impossible to specify the timing and sequence whereby any particular student acquires these key prac¬ 
tices, for the purpose of designing educational assessments, we hypothesize that for most students, the key practices in 
foundational literacy are most relevant to K-4, the key practices in model building are the focus in upper elementary to 
middle school, and the key practices in application are frequently engaged in middle school to high school. We under¬ 
score that the aforementioned timetable is not a claim about when key practices are learned or should be learned, because 
often in high-quality classrooms, higher level key practices are practiced even at the earliest grade levels by way of teacher 
modeling or scaffolded activities. 

For instance, the applied practices often define meaningful contexts within which model building and fundamental 
literacy practices are acquired. Thus, they can and should be introduced, at least in a scaffolded, teacher-supported form, 
from the earliest grades. For example, in shared storybook reading (part of read silently and aloud), an effective kinder¬ 
garten teacher often models comprehension strategies, such as asking questions, making predictions, and paraphrasing, 
that are part of build and share knowledge from text, and he or she may use the occasion to start discussion on some larger 
issue (drawing students into skills needed to participate in the practice discuss and debate ideas). This illustrates a major 
difference between assessments focused on skills and assessments that focus on key practices. By conceiving of literacy 
acquisition as participating in a series of integrated, social practices centered on interactions with print, we accept that 
learning is both social and purposeful. As a consequence, assessment design must take account of this purposeful, social 
nature. 

How Key Practices Connect With the Existing CBAL and Reading for Understanding Frameworks 

At the highest level of abstraction, the existing CBAF ETA framework distinguishes five types of cognitive representation 
that may be involved in a literacy task: 

• Print—orthography, including spelling and decoding. 
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• Verbal—oral and academic language skills. 

• Discourse—skills dealing with the structure of extended texts. 

• Conceptual—abstract reasoning skills, including logic and inference. 

• Social—skills connected with understanding social situations and reasoning about literacy tasks within social con¬ 
texts. 

The CBAL framework also distinguishes three types of cognitive processes: 

• Interpretive—getting meaning from speech or written text. 

• Expressive—communicating meaning by encoding it in speech or written text. 

• Deliberative—forms of strategic, self-conscious reasoning. 

The framework posits that new literacy skills typically require deliberative, strategic thinking, which must then be rou- 
tinized (turned into fluent practice requiring little conscious thought) to support progression to higher levels of expertise. 
Any given task requires some mix of skills across these categories; the categories serve as an analytical tool for identifying 
the specific skills that are required in any particular task. 

The 2011 CBAL framework leaves two other sources of variation among tasks implicit. These sources of variation 
are addressed in more recent work as part of the RFU framework and will be included in future versions of the CBAL 
framework: (a) performance moderators, such as fluency, background knowledge, epistemic beliefs, motivation, metacog - 
nitive awareness, and other factors that reflect differences among individuals; and (b) genre, text complexity, and other 
sources of differences among texts. The former are discussed at length by O’Reilly and Sabatini (2013), and we adopt their 
approach. The latter have been extensively analyzed in other reading and writing frameworks, such as the Programme 
for International Student Assessment (2006), the Programme for the International Assessment of Adult Competencies 
(PLAAC Literacy Expert Group, 2009), and the National Assessment of Educational Progress (Salinger, Kamil, Kapinus, 
& Afflerbach, 2005), and we adopt a similar set of distinctions (see Figure 2). 

Each of the distinctions is an analytical tool — a way to identify dimensions of variation that help define discrete skills 
within the larger space of skills relevant to ELA. By focusing on a particular practice, it is possible both to identify specific 
tasks (such as writing an argument essay or organizing information across multiple texts) and to determine what kinds of 
skills are needed to support that task. 

For example, the task of writing an argument essay involves multiple types of cognitive representation: 

• Social—representing audience and purpose to support reasoning about how well the argument will work to con¬ 
vince that audience. 

• Conceptual—representing the logical structure of arguments to generate new arguments and support reasoning 
about their validity. 

• Discourse—representing the formal structure of an essay to support effective presentation of the thesis, major rea¬ 
sons, and supporting evidence. 

It also involves multiple types of cognitive processes: 

• Interpretation—recognizing the structure of the arguments presented in source materials. 

• Deliberation — reasoning about what arguments will be strongest and most convincing. 

• Expression — actually writing an argument essay of one’s own. 

We also recognize that this specific task involves the following: 

• Presupposes that students have significant background knowledge about the issue to be written about and appro¬ 
priate epistemic beliefs about argumentation (accepting, for instance, that argumentation requires justification and 
is not just a matter of personal opinion). 

• Focuses on producing texts in a relatively small range of genres, such as newspaper articles, letters to the editor, 
and multiparagraph essays, all of which fall squarely within the traditional print realm of linear, continuous texts 
produced for asynchronous reception. 

This kind of analysis helps to identify a series of specific skills relevant to argument that students need to learn, teachers 
need to teach, and assessments need to measure. 
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Medium 


Oral, Print, Digital, Multimedia 



Synchronous 

Asynchronous 

Receptive 

Tickers, breaking headlines, 

news alerts, etc. 

Print documents, Web pages, etc. 

Interactive 

Online chat, text messaging, 

etc. 

Online forums, online comments, letters; 

published commentary, responses, reviews, 

etc. 



Linear 

(intended to be read in a 

set sequence) 

Nonlinear 

(intended to be read in a variety 

of sequences) 

Continuous (intended to be read 

as continuous text, though 

interrupts, such as jumping to a 

footnote, are allowed) 

Traditional print text 

without callouts, 

footnotes, or other 

textual apparatus 

Hypertext, complex documents 

using callouts, footnotes, and 

similar offset elements 

Noncontinuous (intended to be 

read in small pieces; relationships 

are not explicitly signaled with 

transitions but must be inferred 

from content or formatting) 

Bulleted lists, outlines, 

tables, charts, and other 

graphic organizers 

Some types of social media such 

as Twitter feeds; literacy tools 

and textual apparatus such as 

dictionaries and thesauruses, 

menus, indices, and tables of 

contents 


Purpose and 
Genre 


Structure 


Textual 

Complexity 


General Text 
Type 


Literary 


Informational 


Practice in Focus 


Develop and Share 
Stories and Social 
Understandings 


Build and Justify 
Interpretations 


Build and Share 
Knowledge 
Conduct Inquiry 
and Research 


Discuss and Debate 
Ideas 


Propose, Review, 
Recommend, and 
Evaluate 


Draft, Revise, Edit, 
and Publish Texts 


Types of Text 



Fiction 

Nonfiction 

Narrative 

Fiction 

Memoir, 

biography, 

autobiography 

Nonnarrative 

Poetry 

Reflective 

essays 


Book reviews, critical essays, critical commentary, etc. 


Textbooks, encyclopedias, informational articles, other 
expository texts 

Abstracts, search engine results, surveys (including 
newspaper articles surveying research on a field), literature 
reviews, historical narratives or other primary source 
documents, laboratory and research reports, journal articles, 
etc. 


Argument essays, op-eds, letters to the editor, political 
speeches, and other argumentative texts 


Proposals, recommendations, frameworks, project reports, 
white papers, and procedural documents such as manuals 


Critiques, peer reviews, and other forms of written feedback 


Description, sequence, causation, problem/solution, comparison, etc. (primary mode of textual 
organization) 


Style, difficulty of vocabulary, complexity of sentence structure, explicitness of textual structure, 
coherence, etc. 


Figure 2 Elements of textual variation. 
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The final element of the 2011 CBAL framework is a set of more than 40 hypothesized learning progressions (LPs) 
designed to capture the results of the kind of analysis sketched previously and link that analysis to what we know about 
the development of language and literacy skills. Levels in the CBAL LPs map onto specific assessment targets that can 
be measured by distinct item designs. When fully worked out, the sequence of items associated with a CBAL LP defines 
a universe of tasks designed to provide evidence about the development of a specific skill. For many of the CBAL LPs, 
the development of items to match LP levels is still under way, such as those for research and inquiry skills. For other 
LPs, such as those relevant to argumentation and summarization skills, the authors have developed a large pool of items 
illustrating all levels of the progressions and have conducted empirical studies to see how well selected test items recover 
the hypothesized LPs (van Rijn, Graf, & Deane, 2014). 

The structure we have just reviewed is appropriate for item design, because it disciplines item design and forces the test 
designer to define skills explicitly and to sample them systematically. But it does not address other key goals of the CBAL 
research initiative, in particular the idea that assessment should be assessment for and as learning, not just assessment 
of learning. This level is where the concept of key practices enters the picture. Key practices define the social frames that 
make specific literacy tasks meaningful by specifying the high-level goals and activities within which particular skills will 
be deployed. The social framing supplied by a key practice explains why a specific skill is important for teachers to teach 
and helps to specify real-world purposes and contexts that can help motivate students to learn that skill. Or to put it 
another way, the specific LPs listed in the CBAL ELA framework can be grouped into sets that correspond to the skills 
needed to perform well within each of the key practices. Once the LPs have been grouped into sets of skills that mutually 
support participation in those key practices, it becomes obvious why key practices matter for instruction and assessment. 

The Link Between Key Practices and Scenario-Based Assessments (SBAs) 

As practiced in CBAL, SBAs emphasize the importance of capturing the processes and strategies that characterize skilled 
performance, not just a final work product or knowledge state. CBAL SBAs thus draw on the tradition of simulation and 
modeling of realistic tasks and problems that first emerged in such contexts as professional certification (Baker, O’Neill, & 
Linn, 1993; Dillon, Boulet, Hawkins, & Swanson, 2004; Dillon, Clyman, Clauser, & Margolis, 2002; Guliker, Biemans, & 
Mulder, 2009; Leigh et al., 2007; Swanson, Norman, & Linn, 1995) and on the broader tradition of performance assessment 
that emphasizes the importance of creating authentic forms of assessment that more closely match the kinds of perfor¬ 
mances that assessments are ultimately designed to measure (Cummings & Maxwell, 1997; Kane, Crooks, & Cohen, 1999; 
Messick, 1994; Miller & Linn, 2000; Newman & Archbald, 1992; Palm, 2008; Ryans & Frederiksen, 1951; Torrance, 1995; 
Wiggins, 1993). 

SBAs form part of a continuum of innovative assessments that include complex computerized simulations, games, and 
other complex performance tasks; build on cognitive analysis of task requirements; and consider a variety of sources of 
evidence, including data from log files and other process measures (Agostinho, Meek, & Herrington, 2005; Gobert, Sao 
Pedro, Baker, Toto, & Montalvo, 2012; Herrington & Oliver, 2000; Herrington, Oliver, & Reeves, 2003; Mislevy, 2008; 
Mislevy et al., 2004; Mislevy et al., 1999; Rupp et al., 2010; Rupp et al., 2012; Shute et al., 2010; Zane, 2009a, 2009b). They 
differ from more open-ended assessment systems such as games and simulations by adopting a more traditional, linear 
sequence of items—which is, however, mapped onto a meaningful task structure derived from an analysis of the key 
practice. Thus, a transparent relationship exists between the sequence of items used to assess a skill and a more free-form 
task that requires the same combination of skills (i.e., actually performing the integrated key practice). SBAs differ from 
more traditional performance assessments such as are advocated by Darling-Hammond and Adamson (2010) because 
they aim to provide measurement both of the ability to perform well in complex, integrated performance tasks and of key 
component skills necessary for success in that more complex task. 

In the context of the CBAL initiative, SBAs are attractive for reasons similar to those outlined by O’Reilly and Sabatini 
(2013). The key practices provide a catalog of literacy activity-system types as targets for SBA designs. The notion of 
literacy learning as apprenticeship also allows us to design subtasks and items in ways that simulate how good teachers 
model and scaffold such activities. This design in turn enables assessments that provide meaningful evidence of student 
learning that are more actionable for teachers and students. In particular, CBAL SBAs can support the following goals: 

• Increased authenticity (SBAs can promote increased engagement and motivation by supplying a realistic context 
and purpose.) 
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• Positive washback (SBAs can model instructionally valuable strategies and thus encourage better classroom 
practice.) 

• Deeper (but structured) measurement (SBAs can break down complex tasks into manageable steps, provide infor¬ 
mation that can help minimize construct-irrelevant differences in performance due to differences in background 
knowledge, and thus encourage deeper processing of content.) 

• Richer information about student strengths and weaknesses (Profiles of relative performance are more likely to be 
meaningful to teachers and students when the functional relationship among the tasks is clear.) 

Achieving these goals imposes severe constraints on the type and sequence of items that can be assembled to create an 
SBA. In particular, the kinds of SBAs we have sought to create for CBAL ELA have been constrained to meet the following 
requirements: 

• General measurement 

• Individual tasks will measure a range of appropriate skills, sufficient to represent the intended construct. 

• Direct dependencies between tasks will be avoided so as to avoid the reduction in measurement information 
such dependencies tend to cause. 

• Relevance to expert practice 

• Individual tasks correspond to activities that a practitioner might actually perform as part of a single, complex, 
integrated performance. 

• The task sequence corresponds to a sequence of subgoals that a skilled practitioner might address as part of a 
single, complex, integrated performance. 

• Relevance to instruction 

• Individual tasks correspond to instructionally appropriate activities that a teacher might use to teach the 
targeted component skills. 

• The task sequence corresponds to a strategy, or combination of strategies, that a teacher might appropriately 
use to teach students how to successfully perform the final, integrated sequence of tasks independently. 

• Measurement properties 

• Tasks in the sequence should vary in the degree to which they expect unscaffolded, independent performance 
so that responses can be interpreted in terms of student progression from novice toward expert status in the 
skills tested. 

Developing a task sequence that satisfies these constraints is not trivial. Perhaps the greatest challenge is that the sce¬ 
narios that are developed must also satisfy the requirement of scalability. It is important to provide strong reasons to 
believe that the sequence of tasks in the SBA will in fact cover the sequence of tasks needed to undertake a larger class of 
performances of broad educational value. But this situation is precisely where the concept of key practices is relevant. If 
assessment design begins by identifying a limited set of key practices, and continues by identifying scenarios that capture 
core activities in one of those key practices, there is every reason to be confident that the resulting scenario structures will 
generalize and have strong educational value. 

In other words, key practices can be viewed as defining classes of scenarios with high educational value. If the educa¬ 
tional importance of a key practice can be established, and SBAs are developed whose scenario is grounded in that practice, 
there will be a stronger presumptive case that the SBA will measure skills that matter to educators and to students. 

From Key Practices to Scenario-Based Assessment (SBA) Design 

To summarize what we have argued thus far, we contend 

• that we can use the idea of key practices to define ELA constructs in a manner consistent with the Vygotskian idea 
of an activity system; 

• that we can use the structure of a key practice to define a class of scenarios that will be shared across a family of 
SBAs and simultaneously be useful in formative, classroom contexts; 
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• that we can map each task within a scenario onto a set of LPs that define critical skills for that task; and 

• that from there we can use levels in the LP to define levels of performance for which we will seek to gather evidence. 

Once we can specify an SBA to this level of detail, we can obtain an abstract structure that can be turned into a test 
blueprint by drawing on task models that specify the kinds of items needed to provide evidence of the specified levels of 
performance on each LP. 

In the following sections, we examine how this set of ideas plays out in four domains (research and inquiry, early read¬ 
ing, informational reading and writing, and argumentation). These four examples are used to explore how an analysis of 
the ELA construct into key practices enables us to link assessment design explicitly with expert practice and best practices 
in instruction. 

To give a sense of the richness of the kinds of literacy practices to which we are building, we start by discussing research 
and inquiry— a skill that, at the higher levels of performance, presupposes mastery of several other key practices. We then 
examine three other key practices in turn, gradually building up a picture of how we can move from a high-level domain 
analysis to a detailed assessment design, using key practices to design the structure of scenarios and LPs to define the 
specific skills to be assessed. The discussions that follow are intended to be illustrative and will gradually build up more 
and more detail as we move from one practice to the next to create a reasonably rich picture of how assessment design can 
be informed by an explicit model of key practices. Each of the key practices discussed in this report will also be examined 
in greater detail in planned future reports. 


Conducting Research and Inquiry 
Overview: Definition of the Key Practice 

Our definition of the key practice conduct inquiry and research involves mastery of the knowledge, skills, and strategies 
needed to participate in a research community, including the abilities to gather, evaluate, and synthesize information 
from multiple sources; to plan and conduct extended inquiry and experimentation to answer driving questions or solve 
problems; and to present information learned from one’s inquiries in appropriate forms and formats, adapting information 
as needed given the knowledge, interests, and perspectives of one’s intended audience. Engaging in research is primarily 
a process of building knowledge from multiple authoritative sources, where credible resources are used to extend and 
advance one’s thinking in a process of continual idea improvement (cf. Scardamalia & Bereiter, 2006). 

The degree to which an individual will be successful in conducting research is a function of such knowledge, skills, and 
strategies; the nature of the required tasks; and available resources as well as performance moderators (O’Reilly & Sabatini, 
2013), such as prior topic knowledge, metacognitive and self-regulation skills, and epistemic beliefs about the nature of 
knowledge and knowing, which have been empirically demonstrated to influence inquiry practices. The processes of 
evaluation, integration, and communication depend heavily on an individual’s content knowledge as well as knowledge of 
specific disciplinary standards and procedures for producing new knowledge (Goldman, 2004; Goldman & Scardamalia, 
2013; Rouet & Britt, 2011). Development of proficiency with inquiry practices may require attention to supporting learners 
in building their metacognitive skills and strategies and, in addition, explicit modeling of the various literacy and inquiry 
activities that disciplinary experts perform in the service of their professional research practices (cf. Chinn & Malhotra, 
2002; Wineburg, 1991, 1998). This conclusion aligns with perspectives on situated cognition that suggest that learning 
occurs through interacting with others in a social context resembling the real-life conditions in which the learned skills 
will be applied (Brown, Collins, & Duguid, 1989a; Brown, Collins, & Newman, 1989b). This theory also suggests that 
expertise develops through continued participation in communities of practice (Lave & Wenger, 1991) wherein novices 
interact with and learn from experts in the context of their authentic activities. In the CBAL framework, we analyze levels 
of performance in this progression from novice to expert while maintaining the understanding that all of these activities 
are embedded within a larger set of social practices. 

Activity Diagram 

Figure 3 presents a schematic of the key practice conduct inquiry and research, as defined earlier. It is intended to capture 
key activities that literate people might perform if their purpose is to conduct research and inquiry from multiple sources 
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What do I already know? 

What do I need to know? 

How am I going to find out what I need to know? 
Does my understanding of the subject fit the facts? 
Are there other ways to conceptualize my subject? 



What is the current state of discussion in the field? 
What ideas can I contribute that are new? 

Given this focus, how should I present the 
information I have gathered? 

Which ports of it should I include? 

How should I embed those parts in my text? 

How should I document this information? 



Are my sources relevant and useful for my task? 

Are my sources reliable and credible? 

Do my sources agree? 

Can I account for differences between my sources? 
How do their perspectives differ? 

How should I organize the information they provide? 


Figure 3 Activity diagram for the key practice conducting inquiry and research. 

of evidence to advance their understanding of a topic, event, or phenomenon and to apply that understanding to answer 
questions, solve problems, or communicate with others. 

According to this analysis, five major types of activities are included in this activity system, two of which constitute links 
to other key practices that we have identified as important prerequisite skills (at least for the higher proficiency levels). 
These activity types are represented as gray rather than blue shapes in Figure 3. The following subgoals and activities are 
involved in this key practice: 

1. Inquiry and information gathering. This class of activities involves strategies for planning, monitoring, and exe¬ 
cuting information gathering or data collection strategies. It includes strategies for assessing information needs, 
formulating and revising research questions, and generating plans for obtaining reliable evidence that both bears 
on those questions and fills gaps in one’s understanding of the topic. Performance on these kinds of activities is 
perhaps most likely to be associated with constructs like information literacy, information and communication 
technology literacy, and scientific thinking skills. 

2 . Key practice: Building and sharing knowledge. This class of activities corresponds to the competencies needed to 
comprehend and communicate information gained from text, with a specific emphasis on understanding informa¬ 
tional or expository materials. This class includes the activities of activating relevant prior knowledge, reading for 
comprehension, clarifying meanings of words and concepts in the face of comprehension difficulties, consolidat¬ 
ing information from texts with prior knowledge, and the writing skills necessary for organizing and presenting 
information to an audience. Conducting research and inquiry presupposes mastery of informational reading skills 
represented in the building and sharing knowledge key practice, and thus a linkage with this key practice, and the 
subgoals and activities associated with that system, is represented in our model. 

3 . Analysis and synthesis. This class of activities corresponds to strategies for comprehending, evaluating, and con¬ 
solidating information drawn from multiple information resources. The skills required for successful multiple- 
document comprehension (e.g., Goldman, 2011) are most well represented in this set of activities, including skills 
required to comparatively evaluate sources that represent different perspectives or are presented in different for¬ 
mats and to build a coherent understanding of a topic or event that captures both common and unique information 
provided by a set of sources. Performance on these activities is likely to depend on individuals’ topic knowledge, 
understanding of the task, and epistemic beliefs. 
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4. Key practice: Discussing and debating ideas. This class of activities corresponds to the knowledge, skills, and strate¬ 
gies needed for participation in argumentative discourse, which is a critical social component of inquiry, particularly 
in science (Duschl & Osborne, 2002). This includes understanding what is at stake and what interests and values 
stakeholders might bring to an argument, considering the possible positions one might take with respect to a con¬ 
troversial issue and choosing a position to support; critically evaluating the claims, reasons, and evidence others 
use to advance their arguments; and structuring and presenting one’s own arguments in a meaningful way. As 
with building and sharing knowledge, participating in discussion and debate is considered a critical phase within 
the practice of research and inquiry such that conducting inquiry may require researchers to pursue subgoals and 
activities specified in that system. 

5. Communication and presentation of results. This class of activities comprises the writing skills needed to organize 
and present information drawn from multiple sources, particularly informational texts that describe the results of 
research studies or reviews of research results, and to present this information to an audience while avoiding plagia¬ 
rism. Communicating and presenting the results of one’s inquiries also requires, at the highest levels, consideration 
of disciplinary constraints on how one should represent, communicate, and evaluate evidence obtained from other 
sources. 

In Figure 3, each of these five activity types is associated with a set of questions that are designed to illustrate the kinds 
of goals or subgoals that individuals are trying to achieve while engaging in those activities. 

We have represented these activities in a circle, as with the activity diagrams representing other key practices, with 
an implied counterclockwise flow of activities from inquiry and information gathering to building and sharing knowl¬ 
edge, analysis and synthesis, participating in discussion and debate, and communicating and presenting results. However, 
it is important to note that these activities may occur in various sequences and configurations, as is consistent with 
the nonlinear and iterative approaches that characterize experts’ inquiry practices (e.g., Brand-Gruwel, Wopereis, & 
Vermetten, 2005; Rouet & Britt, 2011). Although generating guiding questions is an important early step in the inquiry 
process, we do not mean to suggest that all inquiries begin with the explicit formulation of a question or that every inquiry 
will necessarily require all of these activities. A two-step clockwise sequence from inquiry and information gathering 
to communicating and presenting results might represent a simplistic, uncritical approach to conducting inquiries, in 
which an individual types in a search engine query, quickly scans the first available result, and submits a pasted excerpt 
from the text as his or her response, as characteristic of young students who view research as a process of looking up 
answers (e.g., Bilal, 2000, 2001). This is a relatively impoverished approach to conducting research; depending on the 
complexity of one’s questions, the results one retrieves from an online search may provide incomplete, biased, or inac¬ 
curate answers. In contrast, proficient research and inquiry will include multiple cycles of scanning, reading, evaluating, 
and synthesizing across information sources and repeated cycles of writing, revising, and editing to avoid plagiarism 
or strengthen one’s arguments. Therefore, this activity diagram represents the major activities and activity sequences 
that commonly occur in the practice of conducting research and inquiry but does not define a strictly ordered process 
model. 

To summarize, specifying a model of the key practice conduct inquiry and research in an abstracted, flexible way as 
we have presented affords a range of possibilities for task types and sequences that might be incorporated into an SBA 
of research and inquiry skills; this flexibility is particularly useful given the complex and multifaceted nature of inquiry 
and the need to make principled decisions regarding which skills or activities will be the primary targets of the assess¬ 
ment. An analysis of how this key practice connects to LPs and assessment designs is discussed further by Sparks and 
Deane (2014). 


Reading Silently and Aloud: Early Reading 
Overview: Definition of the Key Practice 

At the center of a literacy practice are activities through which a person is engaged—either independently or in interac¬ 
tion with others — in meaningful communication both orally and through the medium of print (Purcell-Gates, Perry, & 
Briseno, 2011; Street, 2003). The depth of engagement with written texts may vary from, for example, listening to someone 
reading or discussing a book to producing one’s own book. Regardless of the medium or modality, a literacy activity is an 
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Figure 4 The key practice read silently and aloud (early reading). 

intentional, social transaction focused on oral or written communication (Blanchard & Moore, 2010; Dobson & Willinsky, 
2009). Here we focus on literacy activities teachers and parents use to help young children learn to read. 

The key practice read silently and aloud is most salient during the transition from prereading to being able to read grade- 
level texts independently and fluently (Adams, 1990). For most students, this corresponds to the period from kindergarten 
to second or third grade. Learning to read at this stage entails more than the acquisition of decoding and comprehension 
skills; children are inducted into the world of literacy with the help of adults and peers, often through a multitude of 
scaffolded activities (Teale & Sulzby, 1986; Yaden, Rowe, & MacGillivray, 2000). 

By introducing the notion of key literacy practices in the design of early reading assessments, we are able to enrich our 
conception of early reading assessment tasks so that they reflect the social contexts in which literacy practices occur in 
the real world. To this end, we have begun to rethink print-related competencies in the digital age and hypothesized LPs 
for these competencies (Feng, Sabatini, Deane, Sands, & Foley, 2015). In addition, we have begun to conceptualize how 
cultural practices around early literacy can be used as models for assessment design (Feng & Foley, 2015). 

Activity Diagram 

Figure 4 provides an overview of the major activities that students must master as part of the key practice, read silently 
and aloud. 

This diagram includes some activities — activating prior knowledge; comprehending text content; and consolidating, 
clarifying, and conveying information—that draw directly on oral language skills (key practice: communicate by speak¬ 
ing and listening). These same skills, in a more differentiated and text-specific form, underlie more advanced literacy 
practices, such as those needed to understand and produce informational text. But it is important to note that we do 
not presuppose ability to apply those skills to written text, only to use them in oral contexts. The skills most in focus for 
instruction are those that make it possible to read and engage with written texts — decoding (the use of alphabetic script), 
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Figure 5 A classroom instantiation of the key practice read silently and aloud in an early reading classroom. 


word recognition, oral reading (such as shared story reading), and ultimately, silent, individual reading (such as reading 
a novel or a magazine). 

It is also important to note that the individual activities listed correspond to a broad range of real social situations. 
Even such activities as decoding letters and recognizing words have a range of meaningful social contexts in which they 
are realized — and not just in school (imagine, for instance, young parents and children interacting, pointing out words 
on signs, or sounding out unfamiliar words while reading together). There are, similarly, a variety of real social situations 
in which reading aloud takes place (including storybook reading), and of course, one of the points of reading instruc¬ 
tion is to enable students to participate in real-world activities, such as reading fiction to themselves, that involve silent 
reading. 

Note that we specifically claim that most of the activities listed in Figure 4 are relevant to oral and silent reading when 
they are enacted in real social situations. Moreover, we claim that effective classroom instruction recognizes this structure 
and motivates reading by modeling the entire practice, not just isolated skills. Figure 5 presents a modified version of 
Figure 4 focusing on the kinds of learning objectives that teachers and students would set for each activity type in an early 
reading classroom: 

• The process begins with teacher modeling and scaffolding, which help to activate prior knowledge, prepare students 
for interacting with the text, and motivate them for learning. 

• Next, the class works on component skills (such as decoding word recognition) that are necessary for the acquisition 
of basic reading skills. Students’ own language abilities (such as various metalinguistic awarenesses) help to facilitate 
the component skill building. 

• As they develop the ability to recognize words, students work to achieve fluency in oral and silent reading, which 
involve additional skills (such as appropriate intonation in oral reading and the ability to skim and scan in silent 
reading). 

• Understanding is the ultimate goal for reading. Students with competency and fluency in the mechanics of reading 
can focus on understanding what the text actually says, as long as they have sufficient prior oral language skill. 

• Finally, students consolidate, clarify, and convey information by using reading and language skills to revisit the text, 
to engage in discussion with others, and to understand the text’s deeper meaning and relevance to their personal 
lives. This type of activity is often modeled and scaffolded by teachers in Grades K-3. 

• The early reading key practice spans important instructional objectives as specified in the CCSS, including founda¬ 
tional skills in reading, speaking and listening, and language. 
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Figure 6 A high-level description of a prototype assessment design for the early reading key practice. 


From Activity Types to Learning Progressions 

The CBAL ELA competency model (Bennett, 2010; Deane et al., 2013; Feng et al., 2015; Sabatini, Bennett, & Deane, 2011; 
Sheehan & O’Reilly, 2011) offers several lenses to analyze competencies involved in literacy practices, one of which is the 
five modes of cognitive representation. For example, reading requires people to coordinate multiple representations of the 
literacy activities, including representations in the social, conceptual, discourse, verbal, and print dimensions. Not every 
representation is prominent in all assessment tasks. Decoding, for instance, primarily involves print-level skills, whereas 
activating prior knowledge for a picture book is likely to involve social inference based on the situations depicted. We 
can map each of these activities onto various CBAF FPs. The full mapping for this design is discussed by Feng and Foley 
(2015); for present purposes, we will skip over the mapping to FPs to consider other key features of how assessment design 
can work within a key practices conception. 

From Conceptual Analyses to Assessment Design 

The notion of key practices helps to ground our constructs to meaningful literacy practices in which students are expected 
to engage. It also provides a natural frame for generating scenarios for CBAF assessments. Consider the sequence of 
tasks described in Figure 5. These tasks correspond directly to early reading activities that might be readily deployed in a 
kindergarten or early elementary classroom. Each of the tasks also provides an opportunity to measure one or more key 
constructs. This yields a high-level design for an early grade reading assessment. Two aspects of the design are highlighted 
in Figure 6: 

• The design targets not only the basic decoding and reading skills but also discourse and social understanding, which 
at this grade band is often carried out in spoken language and with teacher scaffolding. Although it may be chal¬ 
lenging to design tasks to assess oral language and social understanding, we believe these are important constructs 
at the early stage of reading development. 

• The design calls for a number of innovations in measurement technologies, including the use of tablet computers 
for handwriting input, speech-based responses, and the use of game elements in assessing component skills. 
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'Now we re going to take a picture watk through the book. As 
you look at the pictures, think about what might happen in the 
story. We call that making predictions. ’ 


Picture Walk Example 


"What is an invitation? <W) 

Point to the invitation on the page" 


Figure 7 Illustration of the picture walk activity, designed to activate (and provide, if necessary) students’ background knowledge. 


Each of the activities listed in Figure 6 corresponds to effective classroom practices, while also providing the opportu¬ 
nity to measure important skills. For example, for a class of kindergarten students, the picture walk task may map to the 
following EFA-Fiteracy CCSS: 

• Reading for Foundational Skills, RF.K.4. Read emergent-reader texts with purpose and understanding. 

• Reading: Fiterature, RF.K.7. With prompting and support, describe the relationship between illustrations and the 
story in which they appear (e.g., what moment in a story an illustration depicts). 

• Speaking and Fistening, SF.K. 1. Participate in collaborative conversations with diverse partners about kindergarten 
topics and texts with peers and adults in small and larger groups. 

At the same time, the activities correspond closely to actual classroom practice, which supports the creation of innova¬ 
tive item types that simulate different aspects of the classroom scenario. Figure 7 presents a partial mock-up of one such 
item type. 

To summarize, the early reading key practice is a model for understanding key constructs involved in early reading 
development. It is also a model for designing formative reading assessments for early elementary students in approxi¬ 
mately kindergarten to third grade. We have chosen to focus on activities that simulate high-quality early reading instruc¬ 
tion, in which the teacher models many important literacy practices and scaffolds student learning. These activities cre¬ 
ate a rich context for assessing the skills required to successfully participate in fundamental reading practices in our 
society. 

Building and Sharing Knowledge: Informational Reading and Writing 
Overview: Definition of the Key Practice 

Build and share knowledge is the key practice relevant to understanding and learning from informational text. This prac¬ 
tice presupposes the ability to read silently and aloud while extending many of the skills initially covered by that practice 
to address the demands of using documents to acquire and share factual knowledge (O’Reilly, Deane, & Sabatini, 2014). 
As students develop expertise in building and sharing knowledge, they develop strategic and metacognitive abilities that 
help them manage the process of acquiring information derived from longer and more difficult texts and explaining that 
information to others. As a result, students begin to apply 
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• prereading skills and strategies that enable them to activate background knowledge and set appropriate reading 
goals, 

• reading skills and strategies that help them model content expressed in informational texts, 

• metalinguistic skills and strategies that help them learn new vocabulary, deal with complex syntax, and fill in other 
gaps in comprehension, 

• study skills and strategies that help them analyze complex texts, organize what they have learned, and link that 
knowledge to what they already know, and 

• writing skills and strategies that help them explain and present information that they have acquired while taking 
into account the knowledge and interests of an audience 

Activity Diagram 

The activity diagram in Figure 8 shows the general structure of this key practice, which relates the information provided 
in texts to knowledge activated or acquired by an individual during reading. 

From Activity Types to Learning Progressions; From Learning Progressions to Assessment Designs 

In the prior section on early reading, we have already illustrated how we can map from the abstract structure of a key 
practice onto typical classroom activities. But we have not yet considered how we move from a description of activities to 
a description of skills. This is where prior CBAL work on LPs becomes relevant. In that work, we have identified a set of 
skills based on the cognitive and learning sciences literatures and mapped these skills onto LPs. Each LP can be viewed as 
either an instructional or an assessment target. 

Because the key practice build and share knowledge plays a key role in the development of literacy, a relatively large 
number of LPs are relevant (by our analysis, 17). Figure 9 identifies that inventory. When the major task types from Figure 8 


What are my goals in reading this text(s)? What is 
the author’s purpose in writing the text? What is the 
reader’s purpose in reading it? What will the text 
talk about? How much do I already know? 



apply the concepts? Can 1 explain can I fiS ure out what u reall y means? 

them, to myself? 


Figure 8 Activity diagram for the key practice build and share knowledge. 
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Table 1 Part of the Fine-Grained Progression for Reasons and Evidence 


Level Interpretation 


Expression 


Deliberation 


1 


2 


3 


4 


Identifies reasons people give to 
support a specific point 

Identifies supporting reasons or 
evidence in a written text and 
relates it to the point it supports 

Recognizes and explains the 
relationship between main and 
supporting points and keeps track 
of which evidence supports which 
point 

Identifies specific points in a text 
that are vulnerable to objections 
and counterarguments 


Generates at least one reason to 
support a specific point, in 
sentence form 

Generates multiple reasons to 
support a point, and uses these 
reasons to counter others’ 
arguments in an engaging, 
familiar context 

Builds logical, hierarchically 

structured arguments by selecting 
and arranging reasons and 
evidence to support main and 
subsidiary points 

Writes simple critiques or rebuttals 
that critically provide both 
summaries of and responses to 
other people’s arguments 


Can apply template-based argument-generation 
strategies (such as making lists or filling in a 
pro-con chart) 

Can apply analytical strategies to identify 
information needed to support a point, 
reflecting implicit understanding of common 
argument schemes 

Can evaluate the strength of evidence and 
distinguish sound and unsound arguments by 
recognizing common syllogisms and fallacies 


Can apply critical-question strategies for 
commonly used argumentation schemes to 
generate counterarguments and determine how 
to reinforce specific points 


are combined with the specific LPs in Figure 9, the result is a rich array of task types that can be organized and arranged 
in sequence to define possible scenario structures. 

Figure 10 illustrates this idea in the context of an example assessment structure. The activity diagram in Figure 8 
defines a cycle of activities that can apply, in the first instance, to a single text. In Figure 10, we repeat this cycle three times 
(on Text A, Text B, and Text C). The first time, we draw on a wide range of skills relevant to reading comprehension and 
vocabulary development. As the scenario proceeds, more and more emphasis is placed on consolidation of the information 
learned into a single hierarchical representation that captures critical information about a common topic. At the end of 
the scenario, students are asked to perform a writing task in which they produce a single integrated report that explains 
what they have learned from the source texts. 

It is important to note that Figure 10 presents a simplified version of a scenario structure that has already been imple¬ 
mented in an existing CBAL assessment. It provides a replicable model, because other tests could be devised with similar 
structures. At the same time, the model creates a space for a variety of scenario designs that would, nonetheless, model 
the same underlying practice. 

Note that Figure 10 is still a long way from a full test specification. It does not identify the levels of the LPs to be assessed 
and so is consistent with a range of assessments varying in grade level and difficulty. We will examine something closer 
to a full test specification when we consider the next key practice for which we have more detailed analyses — discussing 
and debating ideas. 

Discuss and Debate Ideas 

Overview: Definition of the Key Practice 

The key practice discuss and debate ideas involves mastery of skills and strategies needed to consider an idea from mul¬ 
tiple perspectives and build arguments to favor one position over another, whether orally (by participating in discussion 
and debate) or in written form (by creating, evaluating, and rebutting written arguments). This key practice targets the 
argumentation skills in CCSS, and it is based on prior CBAL argument-related work (Deane, 2011; Deane et al., 2011; 
Song, Deane, Graf, & van Rijn, 2013). A more detailed report is by Deane and Song (2014). 

Activity Diagram 

Figure 11 presents the activity diagram we have created for this key practice. 
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Genre Differentiation 

Print Cue Sensitivity 

Verbal & Conceptual 

Knowledge Telling 

Determine what sort 

Identify structural 

Association 

Review what one already knows about 

of text one is dealing 

features of the text and 

Scan for repeated or associated 

the topic(s) addressed by the text, 

with, infer readers’ or 

use these to 

words that identify key topics 

possibly writing them out in some form 

author’s purposes, set 

identify/recall key 

and themes. [This learning 

to solidify them in memory and provide 

expectations and goals 

terms and concepts. 

progression (LP) was part of 

an external cue for further reasoning or 

for reading. 


vocabulary development.] 

analysis. [This LP was self¬ 
explanation.] 


Goal Setting and Activating Prior Knowledge 


Understanding the Text [may also be used to support consolidation of knowledge] 


Discourse Fluency & Control 
Maintain sufficient fluency, control 
and coordination of text production, 
and comprehension to support 
efficient processing of the text. 


Outlining 

Explicitly represent the hierarchical structure of 
a text, possibly using some form of graphic 
organizer or other external representation of 
text content. 


Knowledge-Based Inference 
Bridge gaps and specify details by 
making inferences that draw upon 
prior knowledge of the topic. [This LP 
was prediction.] 


Clarifying Meanings 


Word Formation 
Use morphological relationships 
between words to infer the meaning 
and use of unfamiliar vocabulary. 
[This LP was analogy.] 


Definition and Lookup 

Clarify what a word or concept means 

using the reasoning strategies built into 

formal definition; clarify understanding 

of a text using lexical tools such as a 

dictionary 


Multiple Meanings 
Recognize when a word or sentence 
structure has multiple possible meanings; 
infer new word meanings using metaphor, 
metonymy, and other conceptual 
relationships. 


Sentence Context 

Use sentence context to infer what a word or 
phrase must mean in context. 


Logical Analysis 

Clarify the meaning and implications of a statement using sentence form to 
constrain logical reasoning. 


Consolidating Knowledge 


Summary/Main Ideas 
Form a gist understanding of the text 
that captures main and supporting 
ideas; use this gist representation to 
recall information for further use. 


Paraphrase 

Capture a clear understanding of 
what part of a text means by 
recasting the same meaning in 
different words. 


Compare, Contrast, and Organize 
Compare and contrast the discourse structure and 
content of multiple sources on a topic, organize 
information with respect to salient categories or 
goal-driven purposes. 


Conveying Knowledge 


Framing an 
Exposition 
Arrange 
content for 
communication 
to others. 


Review/Preview. Apply Standards for Quality of Informational Text 


Relevance 
Discipline oneself to 
expressing only 
information relevant to 
the topic, purpose, and 
specific ideas in focus. 


Accuracy 
Discipline what is 
said or written to 
avoid conveying 
incorrect 
information. 


Level of Detail 
Discipline the level of 
detail so as to provide 
useful information 
without belaboring the 
obvious. 


Clarity of Expression 
Choose language that 
expresses the information to 
be conveyed clearly but 
concisely. 


Figure 9 Mapping from the diagram for the key practice build and share knowledge to specific targeted skills (corresponding to CBAL 
learning progressions [LPs]). 
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Build and Share Knowledge 


Activate 

Knowledge 


Word 

Associations 


r^c 


Understand 
the Text 


, Summary/ 
Mam Idea 


Clarify 

Meanings 


Definition 
[ and Lookup 


Sentence 

Context 


} 


Summary/ 
Main Idea 


Compare. 

Contrast 

Organize 


y 


>c 


Understand 
the Text 


hi 


> c 


Summary/ 
Main Idea 


4 


Paraphrase 


Consolidate 

Knowledge 


JT 


Compare. 

Contrast, 

Organize 


Summary/ 
Main Idea 








[ Understand 

L the Text 

Consolidate 

Knowledge 



Compare. 

Contrast. 

Organize 


Convey 

Knowiedoe 

I ' 


Framing an 
Exposition 


Figure 10 A scenario structure for the key practice build and share knowledge that maps specific tasks to targeted learning 
progressions. 


r 





Her* should I present up- argument: 
nTia: :riw.vi Is mocs gtfittne and 
logical’ 

How w ill people m uh different 
per:pecInez present their arguments 


Jpp~> fluildimg 


IT ho: e opinion: about this issue master? 
ffhat do people who are interested m das 
issue care about 7 

IF >to ami try.ng to commce? How Mill 1 
commce them? 

fTho are others in mg to commce' Hom 
might they com ince them' 


Ra(&.' (h enll i mn 


• 117 vs do I know about the 
subject* 

• Hhat don't J know ? 

• How can 1 find out more? 

• H7*s indentation is 
relevant’ 

S. J 


Recrons end EwUnce 

• What reasons can I use to 
support »$’ conclusions 7 

• Do 1 hene enough 
evidence to support each 
reason f 

• What counterargument 
do 1 need so address' 


Tqhrjg i ftaia 

• ITTior positions are reasonable' 

• Are drey all clear cod defensible 

• nhatposition shouldI take* 

• How should l focus ad limit nq 
positionf 


Create aad Evahiate 
Argumrnf. 


Figure 11 Activity diagram for the key practice discuss and debate ideas. 


According to this analysis, five major types of activities are involved in this key practice: 

• Understand the stakes. To participate in argumentation, students must understand the stakes, which involves think¬ 
ing about the context and the target audience. The set of skills keyed to this activity is appeal building, a form of 
social reasoning focused on modeling how people are moved to belief and action. In this activity, one may reflect on 
the following questions: What do people care about? Whose opinions about this issue matter? Whom am I trying 
to convince, and how will I convince them? 

• Explore the subject. Students must understand the topic to be able to have a meaningful conversation about it. 
Undoubtedly, shallow knowledge leads to ineffective argumentation. Students need to be aware of what they already 
know about the subject and what they do not know. They should consider how to find out relevant information to 
support their arguments. In its simplest form, exploring the subject reduces to building and sharing knowledge 
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Build and Share 
Knowledge 


Take-l-I-A 

Infers which 
side people are 
taking in an 
argument based 
on the reasons 
or evidence they 
provide 



Create and 
Evaluate 
Arguments 


Reason-2-I-A 

Identifies 
supporting 
reasons or 
evidence in a 
written text and 
relates them to 
the point they 
support 




Reason-4-E-A 

Writes simple 
critiques or 
rebuttals that 
critically 
provide 

summaries of or 
responses to 
other people’s 
arguments 




Participate in 




Argumentatio 
n and Debate 

V 

//' \ 



' Reason--3-E-A 



Frame-3-E-A 

Builds logical, 



Writes multi- 

hierarchically 



paragraph essays 

structured 



with appropriate 

\ arguments by 



transition markers 

’ selecting and 



and other macro- 

arranging reasons 



structural cues. 

and evidence to 



including 

support main and 



appropriately 

subsidiary points 



structured thesis 


Understand 
the Text 

Summarization 


Sum-2-I-A 

Can reliably 
identify a 
statement or 
sentence that 
represents the 
central point, 
theme, or moral 
of a short text, 
and accurately 
distinguish the 
main point from 
salient details. 


0 


Sum-3-I-A 

Can ... identify the 
parts of a text (such 
as thesis and topic 
sentences) that 
express the main 
idea and major 
subpoints. 


>=) 


Sum-3-I-L 

May have difficulty 
... in evaluating 
accuracy or 
recognizing exact 
quotations. 



3 


Sum-2-E-L 

... Summaries ... 
may be extended 
inappropriately by 
including opinions 
and elaborative 
inference 


Sum-3-E-A 

Can create a text 
summary that 
includes both the 
central thesis or 
theme and major 
subpoints, 
accurately 
representing the 
macrostructure of 
the summarized text, 
avoiding reuse of 
exact language from 
the source 


Sum-3-E-A 

Can create a text 
summary that 
includes both the 
central thesis or 
theme and major 
subpoints, 
accurately 
representing the 
macrostructure of 
the summarized text, 
avoiding reuse of 
exact language from 
the source 


and topic sentences 


Figure 12 Mapping from activity system structure to learning progression (LP) level descriptors. 


from texts. More advanced forms of this activity will involve development of skills relevant to another key practice, 
conduct inquiry, and research. 

• Consider positions. Considering positions includes developing one’s own position and understanding alternative 
perspectives. The ideal situation is to start by exploring different lines of arguments to evaluate what positions are 
reasonable and defensible, although students often skip this step. After considering various positions, one should 
take a position and be committed to defending it. 

• Create and evaluate arguments. To defend a position, students must present plausible reasons and evidence as well 
as address counterarguments. They should also evaluate other people’s arguments, for example, identifying unwar¬ 
ranted assumptions that could undermine the logic. A series of questions can help students perform this activity: 
What reasons can I use to support my conclusions? Do I have enough evidence to support each reason? What 
counterarguments do I need to address? These activities target reasons and evidence, a form of abstract, conceptual 
reasoning focused on establishing the truth or validity of statements and arguments. 

• Organize and present arguments. Students must consider how to structure and present each argument in the 
discussion, which requires the skills of framing a case. They need to follow a set of genres and genre conven¬ 
tions that govern how argumentative discourse is organized, whether in informal conversation or in written 
texts. 

An important message conveyed by this activity diagram is that the practice of discussing and debating ideas is more 
than creating arguments. Successful engagement in argumentative discourse requires knowing how to move between 
different activities in this system. Students may not be able to write a good persuasive essay because they do not understand 
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what is really at stake, do not know the subject deeply enough, or stick to an unreasonable position, even if they have some 
clue how to build and present arguments. 

We explained these five types of activities in a sequence, but the actual process of building an argument is flexible and 
fluid. Students can start anywhere in the circle and proceed in any direction between activities. They may even repeat 
activities when needed. For example, students may first take a position on a controversial issue, then figure out that they 
need to know more about the issue to be able to develop strong arguments, and as a consequence move on to the activity 
phase of exploring the subject. Once students are satisfied with the information they have gathered, they can develop 
reasons and give evidence to support a position. After that, students may recognize the need to acknowledge and address 
alternative positions, which could lead to further explorations of the issue. Such a process may continue over many cycles, 
in a dialectical (and dialogic) process in which thoughts about an issue are molded by discussion and reasoning rather 
than persisting unchanged from the beginning to the end of the process. 

From Activity Types to Learning Progressions; From Learning Progressions to Assessment Designs 

In Figure 11, we illustrated how an assessment scenario could be designed by selecting different LPs to cover a range of 
literacy activities. Another possibility is to select specific levels of skill within a single LP. This particular strategy can be 
illustrated using an assessment focused on the key practice discuss and debate ideas. For instance, one of the CBAL LPs 
for argument, reasons and evidence, includes the levels and descriptors listed in Table 1. We can identify tasks that require 
students to exercise higher, or lower, levels in the same progression. One family of CBAL assessments does precisely 
this for two different types of LPs: argument and summary (see Figure 12). The result is still one step short of a full test 
specification because it does not identify the items and item types to be used, but it provides a detailed framework for how 
a class of SBAs that would assess both skills in tandem could be designed. 


Conclusion 

How best to link learning theory and assessment in support of improving instruction is a complex but important problem 
to resolve. Key practices help us make the necessary connections. Perhaps one of the most important features of this 
approach to assessment design is the way it links the structure of an SBA to the intended construct. Several advantages 
follow: 

1. The key practices are by their nature linked to expert performance (which helps define the scenario structure) and 
classroom implementation (because they identify specific tasks that need to be modeled for students), which should 
make it easier to combine sound measurement with instructional utility. 

2. As we define each key practice, it becomes easier to identify and analyze scenarios that instantiate those prac¬ 
tices. A wide variety of scenario structures can be generated, but if they measure similar constructs, that fact will 
be evident when the scenario is represented schematically, using the analytic and conceptual framework defined 
earlier. 

3. When we combine key practices with a detailed analysis of component skills, such as the CBAL LPs, we also pro¬ 
vide a strong foundation for assessment development using an ECD approach, because both the scenario structure 
and the skills measured by each item will be explicitly linked to the construct they are intended to measure or 
support. 

At this point, the kinds of analyses we have exemplified here are in a preliminary stage. We hope in future work to 
explore how an explicit model of this type helps frame very specific research questions about the function of SBAs, includ¬ 
ing evaluations of the effectiveness of the assessment design strategy they represent. 


Notes 

1 See http://elalp.cbalwiki.ets.org/ 

2 Available online at http://www.corestandards.org/ela-literacy 

3 Also see the NCTE definition at http://www.ncte.org/positions/statements/21stcentdefinition 
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